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ABSTRACT 

Background: The hospital standardised mortality ratio 
(HSMR), anchored at an average score of 100, is a 
controversial macromeasure of hospital quality. The 
measure may be dependent on differences in patient 
coding, particularly since cases labelled as palliative are 
typically excluded. 

Objective: To determine whether palliative coding in 
Canada has changed since the 2007 national 
introduction of publicly released HSMRs, and how 
such changes may have affected results. 
Design: Retrospective database analysis. 
Setting: Inpatients in Canadian hospitals from April 
2004 to March 2010. 

Patients: 12 593 329 hospital discharges recorded in the 
Canadian Institute for Health Information (CIHI) Discharge 
Abstract Database from April 2004 to March 2010. 
IVIeasurements: Crude mortality and palliative care 
coding rates. HSMRs calculated with the same 
methodology as CIHI. A derived hospital standardised 
palliative ratio (HSPR) adjusted to a baseline average of 
100 in 2004-2005. Recalculated HSMRs that included 
palliative cases under varying scenarios. 
Results: Crude mortality and palliative care coding rates 
have been increasing overtime (p<0.001), in keeping with 
the nation's advancing overall morbidity. HSMRs in 
2008-2010 were significantly lower than in 2004-2006 by 
8.55 points (p<0.001). The corresponding HSPR rises 
dramatically between these two time periods by 48.83 
points (p<0.001). Under various HSMR scenarios that 
included palliative cases, the HSMR would have at most 
decreased by 6.35 points, and may have even increased 
slightly. 

Limitations: Inability to calculate a definitively 
comparable HSMR that include palliative cases and to 
account for closely timed changes in national palliative 
care coding guidelines. 

Conclusions: Palliative coding rates in Canadian 
hospitals have increased dramatically since the public 
release of HSMR results. This change may have partially 
contributed to the observed national decline in HSMR. 



Measuring the quality of a hospital is an 
important but exceedingly difficult task. 
Different methods of capturing quality have 
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Article focus ^ 

■ The hospital standardised mortality ratio (HSMR), 
a ratio of a hospital's actual to the expected 
number of deaths, is a popular but controversial 
method of gauging hospital quality. 

■ Because patients labelled as palliative are typically 
excluded from the calculation, the HSMR is poten- 

k tially sensitive to changes in coding practices. 

!■ This study assessed whether the release of 
public HSMR data in Canada was correlated with 
changes in palliative care coding, and how such 
changes might have affected HSMR results. ^ 

Key messages 

■ After public release of HSMR data in Canada, 
national HSMR declined while rates of palliative 
care coding increased dramatically. 

■ If palliative cases had been included, rates of 
HSMR decline would have been at least modestly 
attenuated. 

■ Changes in palliative care coding practices likely 
contributed to the observed decline in HSMR. 

Strengths and limitations of this study 

■ This study is based on over 12.5 million cases 
across Canada over 6 years. 

■ The usual weaknesses of a retrospective analysis 
on database information apply. 



been devised, including composite scores of 
compliance with various quality indicators^ 
to the adoption of balanced scorecard tech- 
niques from the business world. ^ No univer- 
sally accepted gold standard exists. 

The hospital standardised mortality ratio 
(HSMR) is a controversial macrolevel tool for 
measuring the quality of hospital care. The 
HSMR is a conceptually simple ratio of the 
observed deaths to expected deaths, multi- 
plied by 100. An institution's number of 
expected deaths is calculated with a regres- 
sion model using national data;^ having an 
HSMR greater than 100 implies having a 
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mortality rate greater than expected for the types of 
patients admitted. Proponents argue that the metric is 
easy to understand, useful for tracking the effect of 
quality improvement initiatives and encourages hospitals 
to explore processes that ctffect patient Sctfety.* ^ 
Opponents dissent that the HSMR has too many meth- 
odological weaknesses and has not been well-validated as 
a useful measure.*' ' Thus, while some countries have 
endorsed and publicly released HSMR results, other juris- 
dictions have felt that it is not a suitable tool for monitor- 
ing quality improvement.^ 

One major concern of the HSMR is the potential for 
'gaming' in which the rules of the calculation are 
exploited to achieve the semblance of a good outcome 
without actually changing care processes. Because calculat- 
ing the expected number of deaths is dependent on how 
patients have been coded, an opportunity exists for a hos- 
pital to alter how sick their patients appear to be.^ In the 
UK, there has been evidence that systematic coding differ- 
ences may affect HSMR results.^" Specifically, recoding a 
patient as palliative can be a simple way to exclude deaths 
that would otherwise have been included in the calcula- 
tion. Since publication of the HSMR in the UK, there has 
been a noted increaise in palliative coding, with two 
trusts increasing the number of deaths labelled as pallia- 
tive to about 50%.^^ In particular, a public inquiry into 
one UK hospital trust highlighted how abrupt changes in 
palliative care coding can quickly alter HSMRs.^'^ 

In 2007, the Canadian Institute for Health Information 
(CIHI) announced plans to release HSMRs for all large 
hospitals in Canada (excluding the province of Quebec) 
participating in its Discharge Abstract Database (DAD) 
programme. ^""^ Results were ultimately released in 
November 2007, and the public relations ramifications 
for hospitals with high HSMRs were severe. ^'^ ^'^ Similar to 
what occurred in the UK,^^ defensive claims were made 
that poor rates were due to misunderstandings in pallia- 
tive coding,^' although CIHI had prepared and released 
palliative care coding guidelines in 2006.^*^ 

Since 2007, HSMRs across Canada have been declining, 
which has been relayed to the public as a sign that releas- 
ing the data has had a salutary effect.^^ ^" The purpose of 
this study was to explore how palliative coding in 
Canadian hospitals has changed since announcing plans 
to release the HSMR, and how that adjustment may have 
affected subsequent results. 

METHODS 

Ethics approval and funding 

This study was approved by both the Lakeridge Health 
and Mount Sinai Hospital Research Ethics Boards. 
Funding was provided by unrestricted continuing 
medical education funds from Lakeridge Health. 

Data 

A request was made to CIHI to access its record-level 
DAD from Fiscal Years 2004-2009 with anonymity for 



hospital and patient identifiers. This database includes 
12 593 329 discharges across 606 hospitals in Canada 
(excluding the province of Quebec) from April 2004 to 
March 2010, which constitutes our entire study popula- 
tion. The DAD includes a 'most responsible diagnosis' 
which is the principal condition responsible for the 
patient's stay in hospital and up to 24 other discharge 
diagnoses typed as comorbidity, secondary, transfer or 
other diagnoses.^^ Details on the DAD are presented 
elsewhere. 

The authors independendy recalculated monthly 
HSMRs using the methodology released by CIHI and 
the same inclusion and exclusion criteria.'^ The CIHI 
HSMR includes inpatient deaths only. A palliative diag- 
nosis was identified with International Classification of 
Diseases-lO-CA code Z51.5; CIHI excludes cases when 
Z51.5 is the most responsible diagnosis code. Total 
number of diagnoses/case was calculated as the sum of 
coded discharge diagnoses/case. Total number of inter- 
ventions/case was calculated as the sum of Canadian 
Classification of Health Interventions codes/ case. Crude 
percentages were calculated as the variable divided by 
the total number of cases each month. Charlson 
comorbidity scores were calculated using the methods 
described by CIHI.^ 

While expected HSMR is based on all hospitals, CIHI 
only releases the HSMRs of hospitals which have at least 
2500 annual cases meeting inclusion criteria. As hospi- 
tals were de-identified, we labelled institutions as having 
publicly released data as those with at leaist 15 000 quali- 
fying HSMR cases over the six study years (ie, an average 
of at least 2500/year) , which was 84 hospitals. 

Statistical analysis 

Data were analysed using SPSS V19. 

Variables visually exhibiting seasonality on plots were 
adjusted using the SPSS multiplicative algorithm func- 
tion, which yields seasonal component factors propor- 
tional to the overall series level. 

To construct a hospital standardised palliative ratio 
(HSPR) , we used the same approach applied to building 
the HSMR.^2 Using data from 2004 to 2005, we con- 
structed a binary logistic regression model to define vari- 
ables and coefficients that would predict the expected 
number of cases with palliative code Z51.5 as the most 
responsible discharge diagnosis (the CIHI HSMR is also 
based on 2004-2005 data). Inclusion criteria were all 
elective or urgent admissions to an acute care institu- 
tion; cases with a discharge disposition of stillborn or 
cadaver were excluded. Final included variables were 
any diagnosis of metastatic cancer, length of stay, age, 
number of interventions and being a medical versus sur- 
gical case. Details on this model are found in the appen- 
dix. The HSPR is interpreted as the observed number of 
cases with palliation as the most responsible diagnosis/ 
expected such numberxlOO. 

To calculate an HSMR which included palliative cases 
(HSMR-i-palliative), we needed to estimate a coefficient 
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value for these records. In the standard CIHI HSMR, the 
most responsible discharge diagnosis is given a coefficient, 
but no similar value was derived in caises identifying palli- 
ation as the main diagnosis. For many cases with palliation 
as the most responsible diagnosis we found that no further 
diagnoses were provided; when further diagnoses were 
coded, it was unclear which to consider the next most 
responsible condition. Therefore, we used three 
approaches to estimate HSMRs that included palliative 
cases. First, we ran a sensitivity analysis in which the pallia- 
tive coefficient would range from the minimum to the 
maximum possible values of the normally included dis- 
charge diagnoses. The purpose of this approach was to 
capture the entire probable range of HSMRs if another 
diagnosis had been entered as the main diagnosis. 
Second, we estimated a palliative coefficient based on the 
expected mortality of patients in Canadian palliative units 
from other studies, ^'^ which is detailed in the appendix. 
In the absence of primary data, this approach provided an 
indirect estimate of what the palliative coefficient should 
be based on existing literature. Third, we reconstructed 
the usual CIHI HSMR binary logistic regression with all 
variables in the same categorisation, but with the addition 
of palliative care as one of the main diagnosis groups. This 
approach provides a coefficient for palliative caises within 
the existing CIHI statistical model. The usual CIHI HSMR 
regression wais also reconstructed with complete elimin- 
ation of the main diagnosis group as a variable. This 
approach used the existing CIHI regression without the 
single variable that differentiated palliative and non- 
palliative CEises. Details of these three approaches are pro- 
vided in the appendix. 

We defined April 2004 to March 2006 as the time 
prior to changes in palliative coding and HSMR publica- 
tion, April 2006 to March 2008 as the time during and 
April 2008 to March 2010 as the time post. The t-tests 
were used to compare the first and last time intervals. 
Corrections for multiple comparisons were not needed 
as comparisons among other different time periods were 
not examined. Ordinary least-squares linear regression 
was used to analyse trends in clinical and demographics 
variables over time. 



RESULTS 

Clinical and demographic characteristics of study 
population and crude mortality and palliative rates 

Table 1 oudines the baseline characteristics of the study 
population and their trends over the study period. 
Inpatients in Canada have been gradually getting older 
with increasing comorbidities. Mean age, mean number 
of diagnoses/patient, mean number of interventions/ 
patient and Charlson scores have all been increasing. As 
expected, the crude proportion of inpatient deaths and 
the crude proportion of inpatients with a palliative code 
as either a most responsible or any discharge diagnosis 
have correspondingly been increasing as well (table 1 
and figure 1 ) . 
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Figure 1 Crude monthly percentage of deaths and 
percentage of cases with palliative care as any diagnosis 
of Canadian inpatients, April 2004 to March 2010 
(n=12 593 329). 



A comparison of the percentage of inpatients with any 
palliative discharge diagnosis between hospitals with and 
without publicly released HSMR data is shown in figure 2. 
The visual trend of eventually more marked palliative 
coding in hospitals with publicly released data is sup- 
ported by descriptive statistics. In 2004—2006, the mean 
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Figure 2 Crude monthly percentage of cases with palliative 
care as any diagnosis, grouped by inpatients in hospitals that 
do (n patients=4 791 714) and do not (n patients=7 801 635) 
have publicly released Canadian HSMR results, April 2004 to 
March 2012. 



percentage of cases coded with a palliative diagnosis was 
1.67% in hospitals with publicly released data and 1.78% 
in hospitals without, for a mean difference of 0.11% (95% 
CI of difference 0.05 to 0.17, p=0.001). In 2008-2010, this 
pattern is reversed. Hospitals with publicly released data 
now have a higher percentage of caises coded with palli- 
ation at 3.40% compared to 3.18% in hospitals without 
(95% CI of difference 0.10 to 0.35, p=0.001). 

Trends in standardised mortality and standardised 
palliative coding rates 

Seasonally-adjusted nation-wide monthly HSMR rates are 
illustrated in figure 3 and the corresponding rates in the 
time periods before, during and after HSMR publication 
are shown in table 2. HSMR rates are significantly lower in 
2008-2010 than in 2004-2006 by 8.55 points ( p<0.001) . 

To compare with the HSMR, we constructed and cal- 
culated an HSPR expressing the ratio of observed cases 
with a most responsible discharge diagnosis of palliation 
to the expected number based on 2004—2005 data. As 
shown in figure 2 and table 2, the HSPR has been 
increasing, and has jumped 48.83 points in 2008-2010 
compared with 2006-2008 (p<0.001). 

The HSMR strongly correlates negatively with the 
HSPR (Pearson r=-0.862, p<0.001). 

Estimating the effect of including palliative cases 
on HSMR 

In the time period 2008-2010, under three palliative care 
scenarios, the HSMR would have decreased by at most 
6.35 points (p<0.001) and may have potentially even 
increased by 2.30 points (p=0.028) (table 2). This is in 




Time 

Figure 3 Seasonally adjusted HSMR (n=3 816 181) and 
HSPR (n=12 593 172) rates for Canadian inpatients, April 
2004 to March 2010. 



4 



Chong CAKY, Nguyen GC, Wilcox ME. BMJ Open 2012;2:e001729. doi:10.1136/bmjopen-2012-001729 



HSMR and palliative coding 



Table 2 Mean monthly HSMR and HSPR for Canadian inpatients, April 2004 to March 2010 

April 2004 to March 2006 April 2006 to March 2008 April 2008 to March 2010 
(95% CI) (95% CI) (95% CI) 

HSMR as calculated by CIHI 99.60 (98.22 to 100.99) 97.96 (96.25 to 99.67) 91 .06 (89.84 to 92.28) 

(total n=3 816 181) 

HSPR (total n=1 2 593 172) 100.72 (99.20 to 102.25) 121.74 (118.38 to 125.11) 149.56 (145.32 to 153.79) 

HSMR including palliative 

cases 

(total n=3 940 586) 
Substituting range of usual 
main 

diagnosis coefficient values 
for palliative cases 

Minimumvalue 119.64 (118.10 to 121.18) 122.28 (120.70 to 123.86) 120.98 (1 19.55 to 122.42) 

Mean value 1 1 5.06 (1 1 3.61 to 1 1 6.51 ) 116.71 (1 1 5.21 to 1 1 8.22) 1 1 3.77 (1 1 2.52 to 1 1 5.03) 

Maximum value 100.43 (99.16 to 101.69) 99.69 (98.28 to 101.10) 94.24 (93.17 to 95.31) 

Using an estimated 106.80 (105.47 to 108.13) 106.96 (105.53 to 108.39) 102.22 (101.12 to 103.33) 

palliative coefficient 

based on the literature 

Reconstructed models 

using the same 

variables as the original 

HSMR except: 

Include palliation as a 100.28 (99.00 to 101.56) 99.52 (98.09 to 100.96) 93.93 (92.83 to 95.04) 

main diagnosis 

Exclude any main 102.85 (101.40 to 104.31) 106.42 (104.87 to 107.98) 105.16 (103.80 to 106.52) 

diagnosis as variable 

The time period April 2006 to March 2008 is counted as the intervention period when there were national changes to palliative care coding 
and public release of HSMR results. 



contrast to the observed HSMR decrease of 8.55. While 
we would caution that the differing calculations make the 
HSMR+palliative statistic not directly comparable to the 
HSMR, one could roughly use these differences to esti- 
mate that the increase in palliative coding contributed to 
at least one quarter of the observed HSMR decline. 

In the final fiscal year 2009, of 84 hospitals with 
publicly released data over the study period, 6 had 
HSMR+palliative scores (when simply adding palliative 
care as a main diagnosis group) that were 10 or more 
points worse than the usual HSMR. The HSPR scores 
for these six hospitals were significantly higher than the 
remaining hospitals (347.4 vs 140.1, p<0.001). 

DISCUSSION 

Across Canada, we demonstrate that since publication of 
the HSMR, hospitals have dramatically increaised the 
number of patients labelled as palliative, a trend that 
may be more marked at sites with publicly released 
HSMR results. This change strongly correlates with the 
witnessed decrease in HSMR. Our results are in keeping 
with the UK experience, in which palliative care coding 
doubled since palliative exclusions began. The findings 
are also consistent with prior Canadian data suggesting 
that inclusion or exclusion of palliative cases can signifi- 
cantly alter HSMRs.^'^ In our study, including palliative 
cases would have at least modestly attenuated the rate of 



HSMR decline. Similarly, in the UK, including palliative 
cases would have changed whether some hospitals were 
labelled as high versus low performing. We suggest our 
results demonstrate that publicly releasing a quality indi- 
cator induces hospitals to improve their scores on that 
metric at least partially by modifying coding practices. 

Accurate coding is an essential part of administrative 
medicine and public health management. Any form of 
coding involves subjectivity, but ethically coding a 
patient as palliative can be a particularly difficult taisk 
requiring clinical judgment and a standardised, consist- 
ent approach.^ In a public inquiry to one UK trust, 
much debate was held on the intentions behind pallia- 
tive care coding changes, although no formal finding of 
fact was made.^^ Regrettably, the close timing between 
CIHI's release of new palliative care coding guidelines in 
mid-2006 and then plans to release the HSMR make it 
difficult to distinguish between the relative contributions 
of these two effects. For example, in the UK, the 2007 
broadening and subsequent 2010 renarrowing of the 
palliative definition was associated with a commensurate 
expansion and retraction in the percent of deaths coded 
under palliation. Changes in national coding guide- 
lines would be expected to change institutional coding 
practices. This is an important limitation to any pre- and 
post-based study, when two closely spaced interventions 
cannot adequately be separated. Also, it is challenging 
to determine how much of the observed increase in 
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palliative coding is due to improved awareness of the 
importance of proper labelling and thus is appropriate 
and ultimately beneficial. Indeed, over the past decade 
there may truly have been an increase in palliative care 
resource utilisation as has been observed elsewhere.^'' If 
our observed increased palliative coding rate is due to 
such a trend, then one might hypothesise the rise 
should correspond with an equally proportionate uptake 
in palliative care services. Such a study is beyond this 
paper's scope, but we speculate it would be difficult for 
the phenomenal rise in HSPR to be entirely accounted 
for by these effects. 

While our findings suggest that a component of 
gaming may exist, it is very important to note that even 
with including palliative cases, HSMR rates are probably 
still declining since 2007, albeit at a slower rate. Debate 
continues over the value and consequences of publicly 
reporting any quality metric.^' Publicly releasing the 
HSMR may very well have fostered adoption of evidence- 
based practices that improve patient satfety and the 
quality of care, as anecdotally reported. However, 
because the HSMR is dependent on the subjective 
coding process, we feel the tool does not accurately quan- 
tify changes in hospital quality. As such, we agree with 
others who claim that as is, the HSMR is not a particularly 
robust instrument.^^ 

Strengths of this study include the size of the study 
population that spans across a nation with a high number 
of pre- and postimplementation observation points. 
Weaknesses include those shared with all non- 
randomised, retrospective observational database 
studies^'' and the aforementioned inherent weakness of 
having a competing intervention (changes in coding 
guidelines) in our analysis. Specific weaknesses for our 
study include being unable to definitively label which 
hospitals had publicly released data. As well, we could not 
provide a definitive coefficient for palliative cases when 
calculating HSMR+palliative scores that would fit into the 
original model. In particular, the literature-based coeffi- 
cient is based on data that is heavily biased towards 
patients with cancer. Our study also focussed solely on 
palliative care coding; evaluating other methodological 
issues such as the effects of possible shifts in comorbidity 
coding,^" inclusion of readmitted patients,'^'' or shifts 
towards out-of-hospital or other-facility deaths^^ '^^ on 
HSMR was beyond the scope of this study; each may have 
had an effect on the observed decline. Also, it is import- 
ant to note that in contrast to other countries, Canada is 
unusual in allowing palliative care to be positioned as the 
primary diagnosis; such a difference may limit extrapola- 
tion of our findings to other nations. 

Society demands transparency and accountability from 
their public hospitals, but identifying methods that 
openly encourage and accurately monitor quality 
remains a major challenge. We contend our findings indi- 
cate that to at least some degree, when publicly pressured 
to show improvement, administrators will seek ways to 



alter metrics in order to account for perceived local idio- 
syncrasies that they feel bias against their institution's per- 
formance. However, 'real' quality improvement more in 
the spirit intended may still occur. Recentiy the English 
Department of Health has adopted use of the Summary 
Hospital-level Mortality Indicator which includes all pal- 
liative care cases and deaths occurring within 30 days of 
discharge;'^'^ and HSMRs with palliative case adjustments 
can be calculated. Further experience should reveal 
whether such evolutions will lead to a truly accurate 
big-dot measure immune to variations in coding practice. 
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